The article "Research by design: Design-based research and the higher degree research student" 
(Kennedy-Clark, 2013) appeared in the Journal of Learning Design Volume 6, Issue 2 in 2013. Two 
years on, Shannon Kennedy-Clark reflects upon her original article. 
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One of my first thoughts upon being asked to revisit this article (Kennedy-Clark, 2013) was to reflect 
upon how much research has been done on design-based research as a methodology. So, in this 
response, I will draw upon some of the main ideas that are coming from recent studies and consider 
this in the context of Higher Degree Research (HDR). When I was first introduced to design-based 
research, it was the methodological approach being used in a grant-funded research project at the 
Centre for Computer Supported Learning and Cognition (CoCo Lab) at the University of Sydney, 
Australia, where we were building an educational virtual world that was based on Harvard’s River 
City. At the time, most of the literature on design-based research was grounded firmly in the learning 
sciences and came from leading researchers in the field, such as Barab and Squire’s (2004) seminal 
text and from research groups such as the Design-Based Research Collective (2003). As my PhD was 
in this field, the approach seemed to provide more opportunities to improve my understanding of the 
study than other, perhaps more traditional, approaches. It also seemed relevant as design research is a 
methodological response to address issues in education rather than being drawn across from other 
research domains. 

The methodology seemed to provide space for the messy and sometimes “wicked” problems that 
are faced by educators in that it harnesses the relative strengths of multidisciplinary teams, long-term 
projects, iterative design, and multiple sources of data. What stood out in this approach from other 
methodological approaches relevant to education, such as action research, was its emphasis on theory 
building as part of the output as well as practical solutions. This emphasis on design to solve problems 
is not a new notion in educational research. For example, Schon (1983) noted that educators are 
designers who design artefacts to solve problems thus implying that design is at the core of education. 
The chapter by Plomp (2007), offered, at the time, the best rationale for using the methodology in my 
own PhD research and is the chapter that I first direct HDR students to who are considering using a 
design approach. Plomp (2007) explained that “the need for a research approach that addresses 
complex problems in educational practice has been argued by researchers in various ‘corners’ of the 
domain of education from the lack of relevance of much educational research for educational practice” 
(p. 9). Further, Plomp’s (2007) discussion of design-research and provision of a range of models is 
relevant to HDR contexts in that it provides students with a visual representation of research studies 
that can translate across to other contexts. For example, Mafumiko’s (2006) dissertation offers a 
practical outline of how design research could be used in a doctoral dissertation circumventing the 
need for multidisciplinary teams by drawing upon panels of experts as part of the design phases; thus 
providing a concrete strategy for HDR students who are working as solo researchers. It is this practical 
explanation of a research design that is of relevance for HDR students who are often struggling with 
complex problems, limited access to resources and working alone. 
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Hence, my motive for writing the article (Kennedy-Clark, 2013) was to share with other 
researchers possible applications of the methodology in the HDR context, particularly in education. 
The approach also provides opportunities for HDR students to revisit and to iteratively build upon 
their study. This provides space for the often inevitable issues associated with HDR studies such as 
insufficient participants or inconclusive or incomplete data. Given that a HDR qualification is a form 
of research apprenticeship, the design approach can provide a platform for HDR students to apply a 
range of data collection tools and methods of analysis which can lead to a better understanding of 
educational design as well as the relative strengths and weaknesses of these techniques. 

What is of note is that, over the past few years, there has been a growing awareness of the need to 
investigate the approach. The literature is not aimed to criticise but more to critically examine the 
approach. This growth in the literature has also resulted in a commentary on the benefits and 
challenges of using this approach in research (see, for example, Anderson & Shattuck (2012); 
McKenny & Reeves, 2013). McKenny and Reeves (2013) pointed out that one of the main aims of a 
design research approach is to generate theoretical understandings that can be of value to others, and 
this premise can also be extended to include how the methodology can be applied. There is still a 
limited amount of research on how this approach can be used in HDR contexts which is perhaps due in 
part to the length of time taken to complete a Masters or Doctoral dissertation and also that the 
methodology is often overshadowed by the outcomes of the study. That is, HDR students are more 
likely to focus on publishing their findings rather than publish about how they did their research. This 
is perhaps why this paper has been well received as it provides several examples of how HDR students 
conducted their studies and built validity and reliability into their design. So while the article is by no 
means exhaustive, it is an overview of the possibilities of the approach. It is also an example of how 
HDR students can present their own methodology for others to consider. In this respect, there is scope 
for further investigation into how HDR students are using this approach in their research. 
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The following presents the article as it appeared in 2013. With the author’s approval, minor edits 
have been made and the referencing updated to APA 6.0. 
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Abstract 

Design-based research lends itself to educational research as the aim of this 
approach is to develop and refine the design of artefacts, tools and curriculum 
and to advance existing theory or develop new theories that can support and lead 
to a deepened understanding of learning. This paper provides an overview of the 
potential benefits of using a design-based research approach in Higher Degree 
Research (HDR) in Education. Design-based research is most often associated 
with conducting research in technology enhanced learning contexts; however, it 
has also been used in the broader field of research in education. A review of six 
theses was undertaken in order to identify how characteristics of a design-based 
research approach were used in Doctoral dissertations. The results of the 
review indicate that the use of expert groups, micro-phases, diverse participant 
groups, and a flexibly adaptive design enabled the researchers to refine and 
improve their research design and their understanding of the problem. 
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Introduction 

This paper provides an overview of design-based research and the Higher Degree Research (HDR) 
experience. While the term design conjures a range of frameworks and applications, in the context 
of this paper it is used in relation to a methodical approach - design-based research. In research in 
the field of the Learning Sciences and technology-enhanced learning, the use of design-based 
research has gained a reputation as being the methodology of choice (Barab, 2006; Barab & 

Squire, 2004; Edelson, 2002; Fishman, Marx, Blumenfeld, Krajcik, & Soloway, 2004; The Design- 
Based Research Collective, 2003). Design-based research lends itself to the field of educational 
research as its underlying premise is to develop the design of artefacts, technological tools, and 
curriculum and to further an existing theory or develop new theories in naturalistic settings that can 
support and lead to an deepened understanding of learning (Barab, Dodge, Thomas, Jackson, & 
Tuzun, 2007; Barab & Squire, 2004; Fishman, et al., 2004). 
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In design-based research, there is a focus on the design process itself at local level. As 
Schoenfeld (2009) explained: “the products of well conducted design experiments are improved 
interventions and improved understandings of the processes that result in their productiveness” 
(para. 18). They are thus productive contributions to the research community. The cyclic and 
iterative processes involved in design-based research are aligned with the authentic design of 
educational environments; hence, there is a natural alignment between design research and research 
in education (Lesh, 2003). In this respect, design-based research is a methodological approach that 
supports an investigation of a learning design. 

An overview of design-based research is provided in this paper and design-based research is 
discussed in a HDR context. Several theses that have used a design-based research approach are 
discussed and the strategies used by the HDR students to ensure validity and trustworthiness of the 
design are also presented. 


Overview of design-based research 

During the 1960s, design research evolved as a recognised field of study. The first generation of 
design theories were heavily enmeshed in technical design; however, criticisms of this perspective 
contributed to viewing design as a problem solving process (Dorst & Dijkhuis, 1995). The view of 
design as a process of reflection-in-action was a result of constructivist influences on the 
explanation of design (Dorst & Dijkhuis, 1995). Schon’s (1983) constructivist-based proposition of 
an alternative epistemology of practice, presented design as a reflective conversation with the 
situation. In this context, problems are actively framed by designers (or researchers, in this case) 
who make “moves” in improving the perceived current situation or problem. In design research, 
there are two main paradigms: design as problem solving and design as reflection-in-action. 

At this stage, it is necessary to differentiate “design” as a research methodology in education 
contexts from the process of design. When design is viewed as a kind of research approach, it 
tends to be done so in a context that values the creation of knowledge (Faste & Faste, 2012). 
“Design-based research” is a research approach that extends existing methods as a means to 
address the issue of linking theory and practice in educational research. The coining of the term 
“design research” in a methodological context is credited to Ann Brown in 1992 (Collins, Joseph, 
& Bielaczyc, 2004). Brown’s (1992) “design research” converged qualitative and quantitative 
operations, collected multifaceted data and focused on in-depth proving of theory. Wang and 
Hannafin (2005) noted that similar and sometimes interchanged terms such as “design 
experiments,” “design research,” “development research,” “developmental research” and 
“formative research” are often grouped with design-based research. While there are differences 
between the approaches, many of the characteristics are shared. In this paper, the term “design- 
based research” will be used. 

Design-based research is an approach that supports the exploration of educational problems and 
refining theory and practice by defining a pedagogical outcome and then focusing on how to create 
a learning environment that supports the outcome (Reeves, Herrington, & Oliver, 2005; Wang & 
Hannafin, 2005). According to Plomp (2007), design-based research is “like all systematic 
educational and instructional design processes - therefore cyclical in character: analysis, design, 
evaluation and revision activities are iterated until a satisfying balance between ideas (‘the 
intended’) and realisation has been achieved” (p. 13). 
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Design-based research is often defined as a series of approaches rather than a single approach 
allowing for the flexibility of the research design (Barab & Squire, 2004; The Design-Based 
Research Collective, 2003). Due, in part, to the adaptability of the design, the approach has been 
used across a range of educational settings as “one of the main motivations behind design-based 
research is to make learning research more relevant for classroom practices” (Reimann, 2010, p. 
37). Design-based research is often at the convergence of design and theory and the design-based 
research framework supports traditional outcome-based evaluation and the importance of design 
(Edelson, 2002). The emphasis is not on refining education practice but on addressing and dealing 
with theoretical issues and questions that arise (Collins, et al., 2004). Systematic evaluation of the 
consecutive research phases or iterations contributes to theory building (Plomp, 2007). The cyclic 
and iterative processes involved in design-based research are in alignment with the authentic 
design of learning environments and theory building (Lesh, 2003). As such, design-based research 
tends to be adopted by researchers who are conducting studies in authentic educational situations 
such as classroom settings in order to generate theory and design relevant to a particular context. 
Three fundamental principles of design-based research are: 

• addressing complex problems in real contexts in collaboration with practitioners; 

• integrating known and hypothetical design principles with technological advances to render 
plausible solutions to these complex problems; and, 

• conducting rigorous and reflective inquiry to test and refine innovative learning environments 
as well as to define new design principles. 

(Reeves, 2006, p. 58) 

The emphasis on naturalistic settings supports the development of contextualised, but sharable, 
theories and cumulative design knowledge in classroom or learning environments. Schoenfeld 
(2009) stated that “properly constructed, a design experiment consists of the creation of an 
instructional intervention on the basis of a local theory regarding the development of particular 
understandings” (para. 9). Hence, the goal of using a design-based research approach is to “build a 
stronger connection between educational research and real world problems” (Amiel & Reeves, 
2008, p. 34). 

Design-based research lends itself to HDR research as research students place themselves in the 
role of instructor and researcher and conduct their investigation in an authentic, localised, context. 
The characteristics of the research design, such as the iterative use of design, evaluation and 
reflection, the use of expert groups and problem and theory refinement, can be used by HDR 
students to develop a more robust research study. 


Three phases of a design-based research study 

Studies that adopt design-based research tend to have three distinct stages: preliminary research, a 
prototyping phase and an assessment phase (Plomp, 2007). In the preliminary research stage, a 
needs and context analysis is undertaken, a review of the extant literature is conducted, and the 
researcher/s develops a conceptual or theoretical framework for the study. The first phase of a 
design-based research study is fairly standard in HDR contexts in that research students will 
develop a proposal based on the articulation of a research problem that is based on a literature 
review and includes a conceptual or theoretical framework. 
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During the second or prototyping phase, which is the iterative design phase, a number of 
iterations of the materials and/or approach are undertaken, with each iteration being a micro cycle 
(micro phase) of the research. Mixed-methods of data collection are used. The combination of data 
collection strategies allows for a more robust understanding of the learning environment (Bannan- 
Ritland, 2003; Brown, 1992; Fishman, et al., 2004; The Design-Based Research Collective, 2003; 
Wang & Hannafin, 2005). Common forms of data include surveys, observations, interviews, 
logbooks, pre- and post-tests and student products (Barab & Squire, 2004; Ketelhut, Clarke & 
Nelson, 2010; Plomp, 2007). Each of these micro cycles is a stand-alone study that may focus on 
fine-tuning a particular aspect of the study with a formative evaluation being the most important 
research activity at the conclusion of each phase. The formative evaluation is aimed at improving 
and refining the materials, approach and theory. 

The final stage is the assessment phase. The purpose of this phase is to conclude how the 
outcome of the investigation meets the pre-determined specifications of solving the problem 
(Plomp, 2007). In this phase, recommendation for future work may be generated. 


Cycles of iteration and prototyping phases 

It is the second phase of a design-based research approach, that is, the cycles of iteration and 
prototyping, which warrants further investigation due to the value this stage offers HDR students. 
For a HDR student, design-based research permits the use of all and any types of data to reach an 
operative or effective design (Gorard, Roberts & Taylor, 2004; Squire, 2004; Wilson, 2004). The 
focus on iteration is not just to evaluate an innovation, for example, hardware or software, but 
rather to produce and refine design principles that can provide guidance for similar research studies 
or development endeavours (Amiel & Reeves, 2008). 

In Figure 1, a model of design-based research illustrates how the iterative cycles which are 
characteristic of design-based research are part of the process of refining the solutions. In this 
figure, the traditional predictive research approach is contrasted with the design research approach. 
The main difference between the two models is that a design-based research approach supports the 
clarification of the problem and the development of design principles and theory refinement 
through a cycle of reflection, evaluation and refinement whereas a predictive approach supports 
hypothesis development and refinement. 
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Predictive research 



Design-based research 



Refinement of problems, solutions, methods, and design principles 


Figure 1. Model of Design-Based Research (Amiel & Reeves, 2008) 


According to Amiel and Reeves (2008): 

The development of design principles will undergo a series of testing and refinement cycles. 

Data is collected systematically in order to re-define the problems, possible solutions, and the 
principles that might best address them. As data is re-examined and reflected upon, new designs 
are created and implemented, producing a continuous cycle of design-reflection-design, (p. 35) 

The reflection on the data gathered in each iteration and the subsequent re-design is aimed at 
refining the product and theory. As Wang and Hannafin (2005) suggested, in a design-based 
research study, data are analysed immediately, continuously and retrospectively and that part of 
this cycle of data collection involves stages, such as a comprehensive literature review coupled 
with the systematic and purposeful implementation of research methods. This iterative process 
leads to the development of design principles, which are then reflected upon and evaluated through 
the refinement of the problem, solutions and methods. 

What this means is that a HDR student can systematically use iterative cycles of design to 
inform subsequent design prior to the final study and assessment phase. This process of refining 
and redefining of the design and the achievement of goals contributes to the development or 
building on of valid theory (Edelson, 2002; Reeves, et al., 2005; Wang & Hannafin, 2005). 
Wadennan’s (2005) generic research design model (Figure 2) demonstrated the dual outcomes of 
design-based research as the practical product and a contribution to theory. Through the cycles of 
analysis, consultation, development, testing, refinement, reflection and evaluation, the principles 
and the solution implementation are revised and refined. The nature of design-based research as an 
educational research approach means that data collection sustained over several research cycles 
will yield a robust body of data to inform design and theory - it also means that HDR students have 
the opportunity to revise their research focus which may enable students to move away from a 
naive understanding to a more nuanced understanding of an educational problem. 
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Figure 2. Wademan’s (2005) generic research model 


Literature search and article selection strategy 

An electronic search of articles from educational databases, such as ProQuest, and university 
Digital Thesis Collections was conducted for theses published between January 2000 and January 
2013. Keywords included in the search were: design research, design-based research, education, 
phases, cycles, and iteration, which are key characteristics of design-based research. In addition, 
the reference lists of published articles on design-based research were also reviewed to identify 
potential theses. The thesis title, abstract, literature review and methodology were reviewed to 
ensure that the study was specific to design-based research in education. Reference lists from each 
thesis were scanned for relevant references on design-based research. Of the original fifteen theses 
that were examined, six were retained for the review. 


Common characteristics of theses using design-based research 

There were several categories for reviewing the theses. Firstly, that the students had clearly 
articulated that they were using a design research or design-based research approach consistent 
with the definition of the characteristics put forward in the literature. Secondly, that the research 
study used phases or cycles in the design of the research for refining the artefact or approach and 
that design was a central element to the research. Thirdly, that the study was undertaken in an 
education setting. The rationale for these categories was that design needed to underpin the intent of 
the research so that it was foregrounded in the literature review, the methodological approach and 
the results. Identifying theses that were all undertaken in education settings meant that it was 
easier to extract the application of the methodology. That is, all of the studies were aimed at 
improving an aspect of an educational problem. Table 1 outlines criteria engaged in HDR studies 
that were embedded into the research design. 
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Table 1. Design-based research categories 


Criteria 

Description 

Examples of criteria in theses 

Micro-phases 

Series of iterative data 
collection phases that test and 
evaluate design. Each cycle in 
the study is a piece of research 
in itself 

Several tests of materials prior 
to main field test that examine a 
different aspect of the design or 
theory 

Expert groups 

The inclusion of several expert 
groups throughout the study to 
evaluate the materials and data 
collection instruments 

Engage with expert groups, such 
as key stakeholders, visiting 
scholars or senior academics to 
review design. Submit research 
stages for peer review in 
conferences 

Different participant groups 

Test materials with a range of 
participants groups 

Engage groups, such as 
teachers, pre-service teachers, 
designers, in the study prior to 
testing with students. Use 
different pilot groups to test 
different prototypes 

Flexibly adaptive 

Ability to take on several roles 
without losing sight of the role 
of researcher 

Use of evolutionary planning 
framework which is responsive 
to field data and experiences as 
acceptable moments during the 
course of a study 


In Table 2, a comparison of the PhD studies is presented. Each of these six theses claimed 
explicitly to have used a design-based research or design research approach. In these theses, several 
micro phases were conducted and numerous iterations of the original design were undertaken. 

There was an array of research contexts and timelines for the data collection cycles. In the case of 
Bower’s (2008) research, the iterations were conducted over three university semesters with unit of 
study cohorts where Bower was the instructor. Whilst in other studies, such as Squire (2004), there 
was more opportunistic sampling as the study was conducted in a classroom, afterschool program 
and a summer vacation program with smaller interventions where Squire was a visiting instructor. 
The research cycles were given a variety of names, such as iterations, cases, phases and micro¬ 
phase; however, they performed the same function and they were used to refine the research 
design. 

The studies all used mixed-methods of data collection. For example, Kennedy-Clark (2012) and 
Bower (2008) both used persistent observation and discourse analysis, while Masole (2011) used a 
baseline study coupled with survey feedback. Other sources of data included: reflective journals, 
pre-and post-tests and semi-structured interviews. The studies drew on expert feedback and, in 
some instances, the feedback was used as checkpoints (Mafumiko, 2006; Masole, 2011). 

Elsewhere, for example, in Squire’s (2004) study, the input from expert groups and peers was built 
into the design as a form of moderation. 
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Table 2. Comparison of PhD theses using design-based research 


Researcher 

Research Area 

Iterations 

Multi-disciplinary 

Data 

Mafumiko 

(2006) 

Researcher 

Design of 
experimental 
chemistry 
curriculum 

Prototype phases 
(used diverse 
participant groups 
including high school 
student, teachers and 
pre-service teachers; 4 
versions of the 
materials) 

Use of expert 
groups 

Mixed-method, 
pre- tests and post¬ 
tests, interviews, 
surveys, and 
observations 

Squire 

(2004) 

Researcher/ 

instructor 

Use of 
commercial 
games in 
secondary 
education 

Cases (used 3 diverse 
student groups) 

Additional 
researcher and 
teacher used for 
observations an 
data collection 
and to elicit 
different 
interpretations 

Mixed-method, 
observations 
logbooks, and 
student products 

Bower 

(2008) 

Researcher/ 

instructor 

Use of web¬ 
conferencing in 
higher 
education 

Iterations (same unit 
of study over 3 
semesters) 

Peer feedback 

Mixed method, 
persistent 
observation, 
reflective journal, 
and multimodal 
discourse analysis. 

Mor (2010) 
Researcher/ 
instructor 

Design in 

technology 

enhanced 

mathematics 

education 

Iterations (used 4 
iterations including an 
initial ‘0’ iteration of 
free forming ideas) 

Additional 
researcher on all 
instruction 
activities. Multi¬ 
disciplinary team 
and on-going 
feedback from 
peers. 

Mixed method, 
design data, 
student 

productions, and 
observations 

Kennedy- 

Clark 

(2012) 

Researcher/ 

instructor 

Use of games 
for inquiry 
learning in 
secondary 
education and 
pre-service 
teacher 
education 

Micro-phases (used 5 
diverse participant 
groups including 
teachers, pre-service 
teachers and high 
school students; 
several iterations of 
the materials) 

Additional 
researcher for 
coding data. 
Ongoing peer 
review and 
feedback 
processes, and 
expert groups 
used. 

Mixed-method. 

Pre- and post-tests, 
interviews, 
document analysis, 
discourse analysis, 
and persistent 
observation 

Masole 

(2011) 

Researcher 

Assessment in 
Agriculture in 
Botswana 
schools 

Phases (used 2 phases, 
the first was a 
baseline survey and 
the second included 4 
prototype phases) 

Used expert 
groups 

Mixed-method, 
Interviews, 
surveys, and 
observation 
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In regards to the choice of data collection strategy, there was consistency in the rationale for the 
selection. For example, observation and “persistent observation” were used because the researchers 
explained that they were directly involved in the interventions and, in this, observation provided a 
rich source of data that could identify subtle shifts in classroom dynamics while persistent 
observation allows the researcher to identify what is relevant to the study and what is not (Lincoln 
& Guba, 1985). Through persistent observation, a researcher can also see how students and 
teachers function, which groups are motivated, which groups struggle, and how the teachers 
interact with the students. Moreover, logbooks and reflective journals were used to keep a record of 
the events and the researcher’s reflections so that changes in options and ideas could be mapped 
across the study. In the case of studies such as Bower (2008) and Squire (2004), persistent 
observation and the maintaining of logbooks and reflective journals formed a significant part of 
their data. 

The point to be made here is that the research students used multiple forms of data and 
feedback from experts to build moderation and validity into their studies. On their own, in a one- 
shot study, each source of data would probably not be of substantial importance to the thesis, but 
in the case of something like Bower’s (2008) reflective journals that were compiled over the course 
of three iterations of the design, a more robust data set is gathered. What design-based research 
offers HDR students is a research approach that has a mechanism for refinement, reflection and 
triangulation over a number of phases of research. These micro interventions can use a number of 
strategies to build reliability and trustworthiness into the design. 


Micro phases and prototyping phases 

The use of micro phases or prototyping phases in design-based research is a strategy to ensure 
reliability of the design before the final field work study. As design-based research aims to 
ascertain if and why a particular intervention works in a certain context, micro research phases 
provide researchers with an opportunity to refine the design and to gain a more informed 
understanding of why an invention may (or may not) work in that context (Plomp, 2007). Micro 
phases involve a series of small scale design studies that result in the subsequent revaluation of the 
materials before the final product is used in a school-based study. The use of micro phases is part 
of what Plomp (2007) referred to as the prototyping stage: “each cycle in the study is a piece of 
research in itself (i.e. having its research or evaluation question to be addressed with a proper 
research design)” (p. 25). Each phase should be presented as a separate study as there may be 
different research questions, population groups, data samples and methods of data analysis. This 
approach was used by Mafumiko (2006) (see Table 2), who undertook a micro-scale investigation 
of improving the chemistry curriculum in Tanzania, and Squire (2004), who conducted three cases 
in the use of the computer game Civilization III with different student groups in different settings 
in order to refine his design. 


Figure 3 shows the progression through Mafumiko’s (2006) study and highlights incremental 
progression through the phases. It is evident that there were four versions of the design prior to the 
final field test in the school-based study. Here, the design was scrutinised by experts and teachers 
to improve the materials used in the final study. 
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Figure 3. Example of research design showing micro phases adapted from Mafumiko (2006), cited in Plomp 

(2007) 


Masole’s (2011) study, in turn, drew on Mafumkio’s design. Figure 4 presents a variation of the 
research design model. 
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Figure 4. Research design showing micro phases adapted from Mafumiko (2006) cited in Masole (2011) 
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While the aim of the micro phases is not to replicate a preceding phase it can be used to ensure 
dependability. For example, Masole (2011) used a Design, Measure, Analyse, Design, Develop 
and Implement (DMADDI) framework. This was spread across the research phases: define, 
measure, analyse were the first phase, and design, develop and implement were the second phase. 
The original caption has been left in the figure as it acknowledges Mafumiko’s (2006) design. The 
use of the micro phases can build dependability into a study. According to Shenton (2004), a 
researcher can establish dependability “if the work were repeated, in the same context, with the 
same methods and with the same participants, similar results would be obtained” (p. 71). Flence, 
each stage of the study should be informed by research on similar studies. 

In addition, to develop the consistency of the approach, the repetition of the phases is 
encouraged in investigation of “all reasonable areas” to ensure that early closure does not occur, 
thus reducing the impact of researcher bias (Lincoln & Guba, 1985). Studies, such as Bower’s 
(2008) built dependability into the research design by being conducted with three similar cohorts 
over a number of semesters. That is not to say that each cycle was repeated per se, but rather the 
preceding phase was used to inform subsequent designs and the refinement of theory. 


Expert groups 

One of the issues that can arise in design-based research by a solo investigator, such as a FIDR 
student, is the occurrence of conflicting researcher roles; that of the designer and developer, the 
facilitator and the evaluator of research. While playing multiple roles can be beneficial in that a 
researcher can understand the whole process, there are, at times, tensions between the roles. Hence, 
it is proposed that it is necessary for HDR students to implement checkpoints during the process to 
ensure that objectivity is maintained. The use of multidisciplinary research teams is seen as a 
strength of design-based research as a greater breadth of understanding can be brought into the 
research environment that from solo research or mono-disciplinary studies (Reeves, et al., 2005; 
Wang & Hannafin, 2005). 

HDR students can draw on the benefits of a multidisciplinary team through the inclusion of 
several expert groups throughout the study to evaluate the materials and data collection 
instruments (surveys, pre-and post-tests and observation schedules) and interrogate the findings 
providing a degree of rigour that may otherwise escape a solo researcher. As the designer and 
developer, solo researchers need to ensure that the instruments and materials are testing what they 
were meant to test. This process of external review should occur prior to the initial data collection 
and any school-based field test. Mafumiko (2006), Masole (2011), Kennedy-Clark (2012) and 
Squire (2004) subjected their designs to the scrutiny of experts. Where possible, the data should be 
coded by a second researcher. The results should also be submitted as conference papers in a 
timely manner to maintain objectivity via a blind peer review process. As the designer and 
developer of the materials, this process of external and internal review maintains the integrity of 
the research. Overall, Cobb, Confrey, diSessa, Lehrer and Schauble (2003) clarify that the size and 
type of research teams depend on the purpose of the research and they explain that modifications to 
the research design may be necessary in a study wherein the researcher is conducting the teaching 
sessions. 
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Diverse participant groups 

The use of prototype phases with a range of relevant participant groups is also recommended for 
two reasons. As Reimann (2010) explained, it is often difficult for ethical and practical reasons to 
conduct lengthy studies in classroom situations. For ethical reasons, school student populations are 
difficult to access. School-based studies are also resource-intensive and running ongoing studies 
with student groups can be difficult for a HDR student, or, in fact, for any researcher. Using a 
range of participant groups in the prototype phases can mitigate issues of accessibility. For 
example, Squire (2004) accessed participants in after-school programs as well as classrooms. 
Masole (2011), Kennedy-Clark (2012) and Mafumiko (2006) used diverse participant groups 
including teachers, pre-service teachers and school students. Moreover, by accessing a range of 
relevant participant groups, such as teachers and pre-service teachers, value can be added to a 
study as they can identify issues with the design prior to the final field test. By conducting studies 
with a range of participant groups, the materials can be critically analysed prior to the final field 
test with a student group. 


Flexibly adaptive research design 

The nature of design-based research necessitates researcher adaptability. The notion of 
evolutionary planning was described by McKenney, Nieveen and van den Akker (2006) as a 
planning framework that is “responsive to field data and experiences as acceptable moments during 
the course of a study” (p. 84). Adaptability, according to Plomp (2007), can be ensured by the 
researcher being prepared to take on the role of designer, advisor and facilitator without losing 
sight of being a researcher. Plomp (2007) also explained that as the research takes place in a real 
world setting, often the wishes and needs of partners may influence the study. Given that design- 
based research takes place in a “real world” context and is based on iterative cycles of design and 
re-design resulting in ongoing changes, it is necessary to implement a planning framework. 

Overall, while the examples here are drawn from educational fields, the focus of the studies was 
diverse including curriculum re-design, science education and maths education. Participant groups 
ranged from school students to teachers. What is of note is that all of the studies used prototype 
phases and several iterations to refine the problem and design used in the study. 

Conclusion 

This paper has focused on design-based research in education; however, there is potential in using 
a design-based research approach across the disciplines. The three phases of a design-based 
research study build reliability into the design by enabling checkpoints that support a HDR student 
in redefining and reflecting on their research as it progresses. It was found in the review of the 
theses that using design-based research allowed for the realisation of promising small-scale 
examples of interventions and the generation of methodological guidelines for the design and 
evaluation of these educational interventions. One benefit raised in the review was that the cycles 
of iteration and evaluation of the design process might actually reduce the overstating of assertions 
and conclusions. 
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The purpose of this paper was not to push design-based research as the way of conducting 
research in HDR contexts; rather, the benefits of this research approach are outlined as means of 
demonstrating how the use of cycles of iteration and prototype or micro phases can be used to 
build reliability and trustworthiness into a research study. One final note on design-based research 
and the criteria presented in this paper is that HDR students often lack the resources to conduct 
large scale research studies and consequently focus on micro studies which may be more 
manageable and achievable. This does not reduce or nullify the value of these contributions to the 
field. However, what it does suggest is that the contributions need to be appreciated on the basis of 
the trustworthiness of the design and the contributions that these studies make to local educational 
contexts and theory building. On the whole, the crucial determinant in a solo or team research 
study is that the individual or team has the expertise and skills to develop the initial design, 
undertake the experiment and undertake a systematic and retrospective analysis of the data. 
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